Animating Lips-Sync Speech Faces with Compact Key-Shapes

Facial animation is traditionally considered as important but tedious work for most applications, because the muscles on the face are complex and dynamically interacting. Although there are several methods proposed to ease the burden from artists to create animating faces, non of them are fast and efficient in storage. In this paper, we introduce a framework for synthesizing lips-sync facial animation by giving a speech sequence. Starting from tracking features on a training video, we first find the representative key-shapes that is important for both image reconstruction and guiding the artists to create corresponding 3D models. The training video is then parameterized to weighting space, or cross-mapping, then the dynamic of features on the face is learned for each kind of phoneme. The proposed system can finally synthesis a lips-sync 3D facial animation in a very short time, and requires very small amount storage to keep the information of the key-shape models and phoneme dynamics.